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My 1991 NASA/ASEE Summer Faculty Fellowship activities at the Langley Research Center 
(LaRC) were directed towards the identification of the opportunities for application of 
Multiple Criteria Decision Making (MCDM) techniques in the Space Exploration Initiative 
(SEI) domain. I identified several application possibilities and proposed demonstration 
application in these three areas: (1) evaluation and ranking of SEI architectures, (2) space 
mission planning and selection, and (3) space system design. Due to page limitations, this 
report describes only the results of my research efforts directed towards the first problem. A 
more detailed report about all three problems would be issued later. 

SEI Architectures 

SEI represents the focused efforts by NASA in meeting President Bush’s challenge to the 
American’s, made on the 20th anniversary of first manned Moon landing, to go "..back to the 
Moon., (and this time) to stay... and a manned mission to Mars." Accordingly, several studies 
have been conducted to define strategies about how this can be accomplished. One of the 
first detailed report discussing scenarios (called "architectures") for Mars missions is the "90- 
day Study" report by NASA( 1 ]; and, the most recent one is the report of the "Synthesis 
Group" set up by Vice-President Quayle[2J. NASA would now be conducting additional 
technical studies to arrive at a baseline SEI architecture. My research was aimed towards 
demonstrating that MCDM methods can assist in this. The following five architectures were 
chosen for evaluation and ranking: 

A : Architecture I (Mars Exploration) of the Synthesis Group Report. 

B : Architecture II (Science Emphasis for the Moon and Mars) of the Synthesis Group 

Report. 

C : Architecture III (Moon to Stay and Mars Exploration) of the Synthesis Group Report. 

D : Architecture IV (Space Resource Utilization) of the Synthesis Group Report. 

E : Modified Reference Architecture of the 90-Day Study Report. 

Evaluation Criteria 

Several researchers from the SEI Office at LaRC participated in the development of a two- 
level, hierarchically structured set of general evaluation criteria. Specific, technical criteria 
were not deemed to be relevant at this early stage of architectural concepts. Table 1 shows 
the developed criteria set. 
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Relative Importance of the Evaluation Criteria 

Many approaches for the development of relative importance weights are availablef31. We 
have used the concept proposed by Saaty for Analytic Hierarchy Process (AHr). Using the 
scale shown in Table 2, SEI researchers first made pairwise comparisons of the major criteria. 
Weights for these major criteria were then computed from these comparisons; see column 2 
of Table 3. Next, they made pairwise comparisons of the sub-criteria under each major 
criterion. Weights from these comparisons and the weights of the major criteria computed 
previously were used to determine the absolute weights of each sub-criterion. Finally, sub- 
criteria with only marginal effects were combined, to reduce the set of the evaluation criteria. 
Table 3 shows the resultant reduced set of evaluation criteria and the corresponding weights. 


Evaluation of the Architectures 


Each of the five architectures were then evaluated by the SEI researchers with respect to each 
t e sub-criterion. The scale in Table 4 was used in these evaluations, and the results are 
shown in Table 5. 


Rank Ordering of the Architectures 

Different rank ordering methods, such as Simple Additive Weighing, ELECTRE and TOPSIS, 
are available. We have used TOPSIS by Hwang and Yoon[3J. TOPSIS uses the Euclidian 
distance of each alternative from an "Ideal Solution" (constructed from the best achieved • 
value with respect to each criterion by any of the alternatives under consideration) and a 
"Negative Ideal Solution" (constructed from the worst criteria values of all the alternatives) to 
compute a closeness measure, C*. The closeness measure for an alternative equals 1 when it 
coincides with the "Ideal Solution" and the measure equals 0 when the alternative coincides 
with the Negative Ideal Solution." Table 6 shows the computed closeness measure values 
for all five of the architectures, and their final ranking. 

Concluding Remarks 

The most meaningful result from this analysis is the wide separation between the top two 
ranked architectures, indicating a significant preference difference between them. It must also 
be noted that the final ranking reflects, to some extent, the biases of the evaluators and their 
understanding of the architectures. 
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Table 1. Evaluation Criteria 


Criterion Sub-criterion 


Explanation 


Maximize the following criteria (i.e., the more the better) 


1. Utility/Benefit 

1.1 Economical 

1.2 Technological 

1.3 Educational 

1.4 Scientific 

1.5 Commercial 

1.6 Synergy 

1.7 Visibility 

1 .8 International 
Cooperation 

1.9 Political 


Potential national economic payoffs 
Technological Payoffs/spinoffs 
Impact on national education system 
Impact on scientific knowledge 
Potential for commercial payoffs 

Positive (synergistic) effect on other SEI/Civilian space projects 

Public visibility and appeal 

Degree of International cooperation that 

may be achieved 

Political appeal and attractiveness 


2. Feasibility 


2.1 Technical 

2.2 Schedule 

2.3 Political 


Soundness and reasonableness of the technological state-of-the-art 
assumptions 

Functionality and reasonableness of the schedule 
Potential for public/congrcss support 


3. Flexibility 

3.1 Launch Date 

3.2 Budget 

3.3 Schedule 

3.4 Mission 

3.5 Design 


Flexibility in launch date 
Flexibility in the required budget 

Flexibility in the overall schedule (resiliency in the schedule) 
Adaptability to changes in the mission goal, focus, etc. 
Robustness of the architectural design 


4. Manageability 


4.1 

4.2 


Developmental 

Operational 


Degree of difficulty in the management of the development efforts 
Degree of difficulty in the management of the mission operations 


Minimize the following criteria (i.c., the less the better) 


5. Risk/Uncertainty 

5.1 Crew Safety 

5.2 Technological 

5.3 Economical 

5.4 Schedule 

5.5 Performance 


Potential risk in crew safety and health 

Risk that the expected technology would not be developed at all or in a 
timely manner 

Economic risk/unccrtainiy (potential, and degree thereof, for the budget 
to increase) 

Robustness of the schedule (level of uncertainty in the schedule) 
Potential uncertainty about achieving the expected performance 


6. Cost 


6.1 Life Cycle Cost Total developmental and operating cost 

6.2 Conflict Potential for detrimental effect on other SEI/Civilian space projects? 
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Table 2. Pairwise Preference Measurement Scale 


/ 

f 


Intensity of 
Importance of 
criterion C over 

criterion C, Definition Explanation 

1 Equal importance C, and Q are equally important 


Weak importance Experience & judgement slightly favor C, over Q 

Essential or strong Experience & judgement strongly favor C, over Q 

importance 


Demonstrated importance C, is strongly favored & its dominance is demonstrated in 
practice 

Absolute importance The evidence favoring C, over Q is of the highest 
possible order of affirmation 

Intermediate values When compromise is needed between two adjacent judgements 
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Table 3 Final Weights of Evaluation Criteria 


Criterion 


Utility/Bcncfit 


Feasibility 


Flexibility 


Weight Sub-criterion 


Scientific 


Visibility 

Political 


Technical 


Schedule 


Political 


Launch Date/ 
Schedule 


Weight 


Relative 


Economical/ 

Technological/ 

Educational 


Absolute 



Mission 


0.6095 


0.0471 


0.1713 


0.0133 


Manageability 

Risk/Unccrtainty 


Crew Safety 

Technological/ 

Performance 

Economical/ 

Schedule 


0.6544 

0.1690 


0.1766 


0.0331 


0.2362 

0.0610 


0.0638 


Life Cycle Cost 0.9000 


0.1214 


Conflict 


38 





Table 4. Architecture Evaluation Scale 


For Cost Attributes 

0 

i 

0 

For Benefit Attributes 

very high 

1.0 

i 

1 

i 

1.0 

very low 

high 

3.0 

i 

1 

1 

i 

3.0 

low 

average 

5.0 

i 

1 

1 

1 

5.0 

average 

low 

7.0 

1 

1 

1 

i 

7.0 

high 

very low 

9.0 

i 

1 

1 

9.0 

very high 


10.0 


10.0 
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Table 5. Architecture Evaluation Matrix 


(Sub) Criterion 


Architecture Evaluation 


A 

B 

c 

D 

Econ/Tcch/Educ 

Benefit 

6 

7 

8 

7 

Scientific 

Benefit 

6 

9 

8 

5 

Visibility 

Benefit 

6 

8 

9 

6 

Political 

Benefit 

6 

7 

6 

7 

Technical 

Feasibility 

7 

5 

6 

3 

Schedule 

Feasibility 

4 

2 

1 

2 

Political 

Feasibility 

5 

5 

1 

5 

Launch/Schcduic 

5 

5 

3 

5 

Flexibility 





Mission 

Flexibility 

7 

5 

i 

8 

5 

Design 

Flexibility 

6 

i 

6 

6 

6 

Manageability 

6 

5 

4 

5 

Crew Safely 
Risk 

6 

6 

4 

6 

T cchn/Pcrformancc 

4 

4 

3 

4 

Uncertainty 





Econ/Schcdule 

Uncertainly 

5 

3 

3 

2 

Life Cycle Cost 

5 

2 

5 

7 

Conflict Cost 

3 

2 

1 

2 


E 

6 

5 

6 
6 
9 


8 

6 

6 

7 

7 

5 
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Table 6. Ranking of the Architerfnrpg 


. ^ 


Architecture 

Separation Measures 

Relative 


(From A*) S jt 

(From A ) S ( . 

C’ioseness 

Q. 

Rank 

A 

0.08135 

0.11308 

0.5816 

2 

B 

0.12002 

0.05481 

0.3135 

4 

C 

0.14060 

0.03552 

0.2017 

5 

D 

0.11353 

0.07253 

0.3898 

3 

E 

0.00724 

0.15107 

0.9543 

1 
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